Protocol conversion of a video stream

ABSTRACT

Aspects of the technology described herein are directed towards systems, methods, and computer storage media for, among other things, converting a video stream being transmitted in a first streaming protocol to a second streaming protocol without transcoding the content communicated in the video stream. For example, the technology described herein may convert an RTP video stream to a non-RTP video stream without transcoding. The technology described herein extracts a plurality of media content from an RTP package and repackages the extracted content into a non-RTP streaming protocol, such as WebRTC or HLS. Moreover, the technology described herein can provide for the synchronization of video and audio data during conversion.

CROSS-REFERENCE TO RELATED DOCUMENTS

The current patent application is a continuation patent applicationwhich claims priority benefit to identically-titled U.S. patentapplication Ser. No. 16/421,621, filed May 24, 2019, which, itself,claims priority to U.S. Provisional Patent Application Ser. No.62/676,153, filed May 24, 2018, entitled PROTOCOL CONVERSION OF A VIDEOSTREAM, and each of the foregoing applications is hereby incorporated byreference in their entireties into the current patent application.

BACKGROUND

Video streaming is a staple of a modern life. Video is regularlystreamed from mobile devices to a plurality of viewers, as embodied byservices such as Facebook Live or Periscope. More traditional videomonitoring devices such as baby monitors and surveillance cameras areroutine sources for video to streaming clients. Finally, video chatting,whether personal or for business, passes streaming video and audiobetween at least two users. These are just some of the many ways inwhich video is streamed from at least one source to at least one viewer.

Substantial numbers of cameras rely on the Real Time Streaming Protocol(RTSP) to control how video information is processed. In the past, manyweb browsers finely permitted plugins to display RTSP video. Claimingvarious concerns, some web browsers such as Mozilla Firefox and GoogleChrome have defaulted to disallow plugins, inherently limiting abrowser-user's ability to view RTSP video. Conventional techniques ofconverting RTSP to plugin-free protocols such as WebRTC and HLS requirestranscoding, a time and resource intensive endeavor.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used in isolation as an aid in determining the scope of the claimedsubject matter.

Aspects of the technology described herein are directed towards systems,methods, and computer storage media for, among other things, convertinga video stream being transmitted in a first streaming protocol to asecond streaming protocol without transcoding the content communicatedin the video stream. For example, the technology described herein mayconvert an RTP video stream to a non-RTP video stream withouttranscoding. The technology described herein extracts a plurality ofmedia content from an RTP package and repackages the extracted contentinto a non-RTP streaming protocol, such as WebRTC or HLS. Moreover, thetechnology described herein can provide for the synchronization of videoand audio data during conversion. Converting a video withouttranscoding, while selectively maintaining video and audiosynchronization can be used to decrease conversion resources anddecrease buffering and lag.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the invention are described in detail below with reference tothe attached drawing figures, wherein:

FIG. 1 is a block diagram of an example operating environment suitablefor implementing aspects of the technology;

FIG. 2 is a diagram depicting an example computing architecture suitablefor implementing aspects of the technology;

FIG. 3 is a diagram depicting a method of converting a media stream froma first streaming protocol to a second streaming protocol, in accordancewith an aspect of the technology described herein;

FIG. 4 is a diagram depicting a method of converting a media stream froma first streaming protocol to a second streaming protocol, in accordancewith an aspect of the technology described herein;

FIG. 5 is a diagram depicting a method of converting a media stream froma first streaming protocol to a second streaming protocol, in accordancewith an aspect of the technology described herein; and

FIG. 6 is a block diagram of an exemplary computing environment suitablefor use in implementing an aspect of the technology.

DETAILED DESCRIPTION

The subject matter of aspects of the technology is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

Aspects of the technology described herein are directed towards systems,methods, and computer storage media for, among other things, convertinga video stream being transmitted in a first streaming protocol to asecond streaming protocol without transcoding the content communicatedin the video stream. For example, the technology described herein mayconvert an RTP (Real-Time Transfer Protocol) video stream to a non-RTPvideo stream without transcoding. The technology described hereinextracts a plurality of media content from an RTP package and repackagesthe extracted content into a non-RTP streaming protocol, such as WebRTCor HLS. Moreover, the technology described herein can provide for thesynchronization of video and audio data during conversion. Moreover, thetechnology described herein can provide for the synchronization of videoand audio data during conversion. The correlated audio and video datamay be configurably synched or unsynched as best-suited for the user'spurpose. Furthermore, the technology can comprise a first buffer fortypical forward-streaming video and a second buffer used for reversevideo playback.

Converting a video without transcoding, while selectively maintainingvideo and audio synchronization can be used to decrease conversionresources and decrease buffering and lag.

Video Stream: as used herein refers to a content delivery method wherevideo content (and optionally audio content and media controls) iscommunicated from a video source to a client device over a networkcommunication session for “just in time” presentation to a user of theclient device. Streaming a video contrasts with downloading a video filein its entirety before watching it. Downloading a video file is adifferent content delivery method. The video content may be presentedthrough the client device by a media player application, such as anInternet browser or media player.

Aspects of the technology described herein can be used to convert a livestream or a recorded stream. As used herein, a live stream refers tovideo content that is transferred from the source camera to a clientdevice within a threshold time period of capturing the video content.Generally, the threshold time period is quite short, such as a fewseconds. In one aspect, the threshold time is less than the length ofthe video content. In other words, the start of a video content iscommunicated to the client before the content capture is completed, forexample, when a sporting event concludes. A recorded stream refers tostreaming a previously created video that has been stored in computerstorage.

Video streaming may be implemented using a series of protocols, not allof which are described herein. First, the video content is encoded usinga content encoding protocol. Generally, the content encoding protocolcompresses the video content for transport over a computer network.Exemplary video encoding protocols include, but are not limited to,H.261, H.263, H.264, HEVC, MPEG-1/MPEG-2. VP8, and VP9. Exemplary audioencoding protocols include, but are not limited to, G.711, G.723. G.726,G.729, GSM, QCELP, MP3, and DTMF.

Video encoding can occur using i-frames, p-frames, and b-frames.I-frames are a complete image, whereas p-frames and b-frames arereferential images that only describe differences between it and anotherimage. A p-frame includes differences from a previous frame. A b-frameincludes differences between both a preceding frame and a subsequentframe. The frames can be packaged in a group of pictures (“GOP”)structure. The GOP structure dictates the order of frames within theGOP. In one example, the GOP starts and ends with an i-frame. Forexample, the GOP structure could be IBBPBBPBBPBBI.

A transport protocol can used to transport the encoded video packetsbetween devices. For example, RTP can run on the UDP transport protocol.

Turning now to FIG. 1 , a block diagram is provided showing an operatingenvironment 100 in which aspects of the present disclosure may beemployed. It should be understood that this and other arrangementsdescribed herein are set forth only as examples. Other arrangements andelements (e.g., machines, interfaces, functions, orders, and groupingsof functions, etc.) can be used in addition to or instead of thoseshown, and some elements may be omitted altogether for the sake ofclarity. Further, many of the elements described herein are functionalentities that may be implemented as discrete or distributed componentsor in conjunction with other components, and in any suitable combinationand location. Various functions described herein as being performed byone or more entities may be carried out by hardware, firmware, and/orsoftware. For instance, some functions may be carried out by a processorexecuting instructions stored in memory.

Among other components not shown, example operating environment 100includes a number of user devices, such as user devices 102 a and 102 bthrough 102 n; a number of video sources, such as video sources 104 aand 104 b through 104 n; server 106; and network 110. It should beunderstood that environment 100 shown in FIG. 1 is an example of onesuitable operating environment. Each of the components shown in FIG. 1may be implemented via any type of computing device, such as computingdevice 600, described in connection to FIG. 6 , for example. Thesecomponents may communicate with each other via network 110, which mayinclude, without limitation, one or more local area networks (LANs)and/or wide area networks (WANs). In exemplary implementations, network110 comprises the Internet and/or a cellular network, amongst any of avariety of possible public and/or private networks.

It should be understood that any number of user devices, servers, anddata sources may be employed within operating environment 100 within thescope of the present disclosure. Each may comprise a single device ormultiple devices cooperating in a distributed environment. For instance,server 106 may be provided via multiple devices arranged in adistributed environment that collectively provide the functionalitydescribed herein. Additionally, other components not shown may also beincluded within the distributed environment.

User devices 102 a and 102 b through 102 n can be client devices on theclient-side of operating environment 100, while server 106 can be on theserver-side of operating environment 100. Server 106 can compriseserver-side software designed to work in conjunction with client-sidesoftware on user devices 102 a and 102 b through 102 n so as toimplement any combination of the features and functionalities discussedin the present disclosure. This division of operating environment 100 isprovided to illustrate one example of a suitable environment, and thereis no requirement for each implementation that any combination of server106 and user devices 102 a and 102 b through 102 n remain as separateentities.

User devices 102 a and 102 b through 102 n may comprise any type ofcomputing device capable of use by a user. For example, in one aspect,user devices 102 a through 102 n may be the type of computing devicedescribed in relation to FIG. 6 herein. By way of example and notlimitation, a user device may be embodied as a personal computer (PC), alaptop computer, a mobile or mobile device, a smartphone, a tabletcomputer, a smart watch, a wearable computer, a personal digitalassistant (PDA), an MP3 player, global positioning system (GPS) ordevice, video player, handheld communications device, gaming device orsystem, entertainment system, vehicle computer system, embedded systemcontroller, remote control, appliance, consumer electronic device, aworkstation, or any combination of these delineated devices, or anyother suitable device where notifications can be presented.

Video sources 104 a and 104 b through 104 n may comprise video sourcesand/or video systems, which are configured to make video available toany of the various constituents of operating environment 100, or system200 described in connection to FIG. 2 . (For example, in one aspect, oneor more video sources 104 a through 104 n provide (or make available foraccessing) protocol transfer component 280 of FIG. 2 .) Video sources104 a and 104 b through 104 n may be discrete from user devices, such aphone, laptop, head mounted display, or tablet having a camera. In oneaspect, one or more of video sources 104 a through 104 n comprises oneor more standalone cameras, such as security cameras, baby monitoringcameras, doorbell cameras, and the like.

Operating environment 100 can be utilized to implement one or more ofthe components of system 200, described in FIG. 2 , including componentsfor receiving a video stream in a first protocol, decoding the firstprotocol to extract video packets encoded according to a first encodingprotocol, and repacking the video packets into a second transportprotocol without transcoding. Referring now to FIG. 2 , with FIG. 1 , ablock diagram is provided showing aspects of an example computing systemarchitecture suitable for implementing an aspect of the technology anddesignated generally as system 200. System 200 represents only oneexample of a suitable computing system architecture. Other arrangementsand elements can be used in addition to or instead of those shown, andsome elements may be omitted altogether for the sake of clarity.Further, as with operating environment 100, many of the elementsdescribed herein are functional entities that may be implemented asdiscrete or distributed components or in conjunction with othercomponents, and in any suitable combination and location.

Example system 200 includes network 110, which is described inconnection to FIG. 1 , and which communicatively couples components ofsystem 200 including user-data video source 204, stream converter 280,and user device 260. Stream converter 280 (including its components 282,284, 286, 288, 290, and 292), user device 260 (including its components262, 264, 266, 268, 270, and 272), video source 204 (including itscomponents 206, 208, 210, 220, 222, 224, 226, 228 and 230), may beembodied as a set of compiled computer instructions or functions,program modules, computer software services, or an arrangement ofprocesses carried out on one or more computer systems, such as computingdevice 600 described in connection to FIG. 6 , for example.

In one aspect, the functions performed by components of system 200 areassociated with one or more applications, services, or routines. Inparticular, such applications, services, or routines may operate on oneor more user devices (such as user device 102 a), servers (such asserver 106), may be distributed across one or more user devices andservers, or be implemented in the cloud. Moreover, in some aspects,these components of system 200 may be distributed across a network,including one or more servers (such as server 106) and client devices(such as user device 102 a), in the cloud, or may reside on a userdevice such as user device 102 a. Moreover, these components, functionsperformed by these components, or services carried out by thesecomponents may be implemented at appropriate abstraction layer(s) suchas the operating system layer, application layer, hardware layer, etc.,of the computing system(s). Alternatively, or in addition, thefunctionality of these components and/or the aspects of the technologydescribed herein can be performed, at least in part, by one or morehardware logic components. For example, and without limitation,illustrative types of hardware logic components that can be used includeField-programmable Gate Arrays (FPGAs), Application-specific IntegratedCircuits (ASICs), Application-specific Standard Products (ASSPs),System-on-a-chip systems (SOCs), Complex Programmable Logic Devices(CPLDs), etc. Additionally, although functionality is described hereinwith regards to specific components shown in example system 200, it iscontemplated that in some aspects functionality of these components canbe shared or distributed across other components.

Continuing with FIG. 2 , video content 201 is communicated from thevideo source 204, to network 110, to the stream converter 280, back tonetwork 110, and finally to the user device 260. The stream converter280 converts the stream content from a first streaming protocol to asecond streaming protocol without transcoding media content. Conversionof a video stream may be necessary when a user device does not have amedia application that is compatible with a particular streamingprotocol. In this case, the stream converter 280 may be used to allowthe video source 204 to stream media content to user device 260 byconverting the video stream from a first protocol to a second protocol.

The video source 204 may be similar to the video sources 104(a)-104(n)described previously. For example, the video source 204 could be acamera, such as a security camera or baby monitor, that is livebroadcasting content into a video stream. As another example, the videosource could be a media server that streams recorded content to one ormore user devices.

The video source 204 includes a camera 206, a microphone 208, a videocontent 210, a video codec 220, audio content 222, an audio codec 224, acommunication session negotiator 226, a packet generator for a firststreaming protocol 228, and a stream controller 230.

The video content 210 is generated by the camera, in the example shownin FIG. 2 . It may be buffered or otherwise stored briefly as it isprocessed for input into the video stream. The raw video content 210 maybe captured in any number of different video formats. However, it may beencoded into a content protocol suitable for use in a video stream.Suitable content protocols will typically compress the video data.

The video codec for a first content protocol 220 converts the raw dataof video content 210 into a first content protocol, such as H.261,H.263, H.264, HEVC, MPEG-1/MPEG-2, VP8, and VP9. The result of theencoding work done by the video codec 220 can be a series of videoframes. The video frames may be grouped into a GOP of frames. The GOPcan include i-frames, b-frames, and p-frames.

The audio content 222 may be captured by the microphone 208 at the sametime video is captured by the camera 206. Like the video content, theaudio content 222 may be stored briefly for processing.

The audio codec 224 is for a second content protocol for audio andconverts the raw audio captured by the microphone 208 into an audiocontent format consistent with the second content encoding protocol.Exemplary audio encoding protocols include, but are not limited to,G.711, G.723, G.726, G.729, GSM, QCELP, MP3, and DTMF.

The communication session negotiator 226 establishes a communicationsession between the video source 204 and the user device 260. Thecommunication session may include the stream converter 280.Alternatively, separate communication sessions may be establishedbetween the video source 204 and the stream converter 280 and a separatesession between the stream converter 280 and the user device 260.

The packet generator for a first streaming protocol 228 takes the audiocontent and the video content and packages it into data packetsconsistent with the first streaming protocol, for example, RTP. Theaudio content may be packaged separately from the video content. Forexample, packet generator may package the media content into RTPpackets.

The stream controller 230 can respond to control signals, such as thoseprovided over the RTSP, to alter the media presentation. For example,the stream controller 230 can pause the stream, stop the stream,fast-forward the stream, and the like.

The stream converter 280 includes an extractor 282 compatible with thefirst streaming protocol, video buffers 284, audio buffers 286, signalengine 288, a packet generator 290 that is compatible with a secondstreaming protocol, and a quality engine 292.

The extractor 282, which is compatible with the first streamingprotocol, receives the data packets, such as RTP packets, and extractsthe media payload. The extracted video payload will be in the firstcontent protocol and the audio content will be in the second contentprotocol.

The video buffers 284 are located between the extractor 282 and thepacket generator 290. The video buffer 284 can store video frames fromthe extractor 282 and perform various functions on them, such asreordering the frames according to a timestamp, sequence number, orother ordering mechanism.

The audio buffers 286 are located between the extractor 282 and thepacket generator 290. The audio buffer 286 can store audio content fromthe extractor 282 and perform various functions on them, such asreordering according to a timestamp, sequence number, or other orderingmechanism. The video buffers 284 and audio buffers 286 may work togetherto synchronize audio and video content.

The signal engine 288 receives control signals from the user device 260and converts them to a control signal that is compatible with the videosource 204. For example, a control to start, stop, pause, fast-forward,rewind, skip, or the like may be converted from one format to anotherfor consumption by the stream controller 230. Likewise, variousacknowledgments, responses, or instructions from the stream controller230 can be converted by the signal engine 288 and communicated to theuser device 260.

The packet generator 290, which is compatible with a second streamingprotocol, repackages the extracted audio and video content into packetscompatible with the second streaming protocol. The repackaging is donewithout transcoding the audio or video content. In other words, theaudio and video content is packaged into the second streaming protocolwhile remaining in the same content protocol, which in this case is thefirst content protocol for video and second content protocol for audio.

The quality engine 292 converts session data associated with the firststreaming protocol to session data associated with a second streamingprotocol. Different streaming protocols may use a streaming sessionchannel to control or monitor the quality of the streaming session. Forexample, the RTSP protocol uses the RTCP protocol to control quality ofservice (QoS) factors.

In one aspect, RTCP controls are received by the quality engine 292 andconverted to quality messages that can be sent to the user device 260.Similarly, the quality engine 292 can receive quality messages from theuser device 260, translate them, and send them on to the video source204. This allows the video source to adjust streaming characteristicsbased on the entire route between the video source 204 and the userdevice 260, instead of just between the stream converter 280 and thevideo source 204.

The user device 260 includes a media application 262, a controlinterface 264, the communication session negotiator 266, a video codecfor a first content protocol 268, and a packet extractor compatible witha second streaming protocol 270, an audio codec compatible with thesecond content protocol 272.

The media application 262 is a computer program capable of playingmedia, such as music or videos. Many different media applications exist.Some work with multiple streaming and video formats, while others areonly able to work with a particular protocol. Many media applicationinclude controls similar to those found on a DVD player. For example,the media application can include play, stop, pause, fast-forward, skip,rewind, and the other controls.

The control interface 264 provides an alternative control interface forthe user to control the media presentation on the media application 262.The control interface 264 can be used instead of controls provided bythe media application 262. As mentioned, some streaming protocolsinclude a control or signaling channel, such as RTSP. It may bepreferable in some implementations to maintain the same control signals,rather than converting from one signal protocol to another. The controlinterface 264 allows the user to communicate with the video source 204,which will control the media presentation.

The communication session negotiator 266 helps establish communicationssessions between components, as needed.

A packet extractor compatible with a second streaming protocol 270removes the media payload, video and/or audio, from the data packets.For example, the payload could be removed from WebRTC packets.

A video codec for a first content protocol 268 converts the compressedvideo frames to a format that can be consumed by the media application262 to present the video.

An audio codec compatible with the second content protocol 272 is usedto remove the audio payload from the second streaming protocol datapacket.

Turning now to FIG. 3 , a method 300 for converting a streaming videofrom a first video streaming protocol to a second video streamingprotocol is provided, in accordance with an aspect of the technologydescribed herein. Method 300 may be performed by stream converter 280.

At step 310, a plurality of data packets encoded in the first videostreaming protocol are received from a computer network. The pluralityof data packets are part of a communication session that transmits avideo content from a video source to a client device. The packets may bereceived by a stream converter 280. The stream converter may be locatedin a data center. The video source may be a camera or device having acamera, such as a doorbell. The computer network may be the Internet.The client device could be a smart phone, PC, TV, tablet, or the like.

At step 320, a plurality of video frames are extracted from theplurality of data packets. The plurality of video frames are encoded ina first content protocol, such as H.261, H.263, H.264, HEVC,MPEG-1/MPEG-2, VP8, and VP9. The video frames may be grouped into a GOPof frames. The GOP can include i-frames, b-frames, and p-frames. Whenextracted the frames remain in the first content protocol, such as VP8.

At step 330, the plurality of video frames are encoded in the secondvideo streaming protocol without transcoding the plurality of videoframes to form a new plurality of data packets encoded according to thesecond video streaming protocol. In other words, the video frames remainin the same content protocol, but are wrapped in a differentcommunication protocol. For example, the video frame encoded in the VP8protocol may be extracted from a data packet encoded in RTP and thenencoded in a WebRTC data packet without changing the VP8 video frame.

Prior to encoding, the frames may be buffered and subjected to variousquality improvement processes. For example, the frames or GOPs could bereordered into a temporally correct order. Missing frames could beidentified and requested from the video source, if not present.

At step 340, the new plurality of data packets to the client device areoutput over the computer network.

Turning now to FIG. 4 , a method 400 for converting a streaming videofrom a first video streaming protocol to a second video streamingprotocol is provided, in accordance with an aspect of the technologydescribed herein. Method 400 may be performed by stream converter 280.

At step 410, a plurality of data packets encoded in the first videostreaming protocol are received from a computer network. The pluralityof data packets are part of a communication session that transmits avideo content from a video source to a client device. The packets may bereceived by a stream converter 280. The stream converter may be locatedin a data center. The video source may be a camera or device having acamera, such as a doorbell. The computer network may be the Internet.The client device could be a smart phone, PC, TV, tablet, or the like.

At step 420, a plurality of video frames are extracted from theplurality of data packets. The plurality of video frames are encoded ina first content protocol, such as H.261, H.263, H.264, HEVC,MPEG-1/MPEG-2, VP8, and VP9. The video frames may be grouped into a GOPof frames. The GOP can include i-frames, b-frames, and p-frames. Whenextracted the frames remain in the first content protocol, such as VP8.

At step 430, a plurality of audio frames are extracted from theplurality of data packets. Exemplary audio encoding protocols include,but are not limited to, G.711, G.723, G.726, G.729, GSM, QCELP, MP3, andDTMF.

At step 440, the plurality of video frames and the plurality of audioframes are encoded into the second video streaming protocol withouttranscoding the plurality of video frames to form a new plurality ofdata packets encoded according to the second video streaming protocol.In other words, the video frames remain in the same content protocol,but are wrapped in a different communication protocol. For example, thevideo frame encoded in the VP8 protocol may be extracted from a datapacket encoded in RTP and then encoded in a WebRTC data packet withoutchanging the VP8 video frame.

Prior to encoding, the frames may be buffered and subjected to variousquality improvement processes. For example, the frames or GOPs could bereordered into a temporally correct order. Missing frames could beidentified and requested from the video source, if not present.

At step 450, the new plurality of data packets are output over thecomputer network to the client device.

Turning now to FIG. 5 , a method 500 for converting a streaming videofrom a first video streaming protocol to a second video streamingprotocol is provided, in accordance with an aspect of the technologydescribed herein. Method 500 may be performed by stream converter 280.

At step 510, real-time transport protocol RTP (Real-Time TransportProtocol) data packets that are part of a communication session betweena video source and a client device are received. The packets may bereceived by a stream converter 280. The stream converter may be locatedin a data center. The video source may be a camera or device having acamera, such as a doorbell. The computer network may be the Internet.The client device could be a smart phone, PC. TV, tablet, or the like.

At step 520, a plurality of video frames are extracted from the RTP datapackets. The plurality of video frames are encoded in a first contentprotocol, such as H.261, H.263, H.264, HEVC, MPEG-1/MPEG-2, VP8, andVP9. The video frames may be grouped into a GOP of frames. The GOP caninclude i-frames, b-frames, and p-frames. When extracted the framesremain in the first content protocol, such as VP8.

At step 530, the plurality of video frames are encoded in WebRTC packetswithout transcoding the plurality of video frames, to form WebRTC datapackets.

At step 540, the WebRTC data packets are communicated to the clientdevice.

With reference to FIG. 6 , computing device 600 includes a bus 610 thatdirectly or indirectly couples the following devices: memory 612, one ormore processors 614, one or more presentation components 616, one ormore input/output (I/O) ports 618, one or more I/O components 620, andan illustrative power supply 622. Bus 610 represents what may be one ormore busses (such as an address bus, data bus, or combination thereof).Although the various blocks of FIG. 6 are shown with lines for the sakeof clarity, in reality, these blocks represent logical, not necessarilyactual, components. For example, one may consider a presentationcomponent such as a display device to be an I/O component. Also,processors have memory. The inventors hereof recognize that such is thenature of the art and reiterate that the diagram of FIG. 6 is merelyillustrative of an exemplary computing device that can be used inconnection with one or more aspects of the present technology.Distinction is not made between such categories as “workstation,”“server,” “laptop,” “handheld device,” etc., as all are contemplatedwithin the scope of FIG. 6 and with reference to “computing device.”

Computing device 600 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by computing device 600 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable media may comprisecomputer-storage media and communication media.

Computer-storage media includes both volatile and nonvolatile, removableand non-removable media implemented in any method or technology forstorage of information such as computer-readable instructions, datastructures, program modules, or other data. Computer storage mediaincludes, but is not limited to, RAM, ROM, EEPROM, flash memory or othermemory technology, CD-ROM, digital versatile disks (DVDs) or otheroptical disk storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed bycomputing device 600. Computer storage media does not comprise signalsper se.

Communication media typically embodies computer-readable instructions,data structures, program modules, or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media, such as awired network or direct-wired connection, and wireless media, such asacoustic, RF, infrared, and other wireless media. Combinations of any ofthe above should also be included within the scope of computer-readablemedia.

Memory 612 includes computer storage media in the form of volatileand/or nonvolatile memory. The memory may be removable, non-removable,or a combination thereof. Exemplary hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 600includes one or more processors 614 that read data from various entitiessuch as memory 612 or I/O components 620. Presentation component(s) 616presents data indications to a user or other device. Exemplarypresentation components include a display device, speaker, printingcomponent, vibrating component, and the like.

The I/O ports 618 allow computing device 600 to be logically coupled toother devices, including I/O components 620, some of which may be builtin. Illustrative components include a microphone, joystick, game pad,satellite dish, scanner, printer, wireless device, etc.

The I/O components 620 may provide a natural user interface (NUI) thatprocesses air gestures, voice, or other physiological inputs generatedby a user. In some instances, inputs may be transmitted to anappropriate network element for further processing. An NUI may implementany combination of speech recognition, touch and stylus recognition,facial recognition, biometric recognition, gesture recognition both onscreen and adjacent to the screen, air gestures, head and eye tracking,and touch recognition associated with displays on the computing device600. The computing device 600 may be equipped with depth cameras, suchas stereoscopic camera systems, infrared camera systems, RGB camerasystems, and combinations of these, for gesture detection andrecognition. Additionally, the computing device 600 may be equipped withaccelerometers or gyroscopes that enable detection of motion. The outputof the accelerometers or gyroscopes may be provided to the display ofthe computing device 600 to render immersive augmented reality orvirtual reality.

Some aspects of computing device 600 may include one or more radio(s)624 (or similar wireless communication components). The radio 624transmits and receives radio or wireless communications. The computingdevice 600 may be a wireless terminal adapted to receive communicationsand media over various wireless networks. Computing device 600 maycommunicate via wireless protocols, such as code division multipleaccess (“CDMA”), global system for mobiles (“GSM”), or time divisionmultiple access (“TDMA”), as well as others, to communicate with otherdevices. The radio communications may be a short-range connection, along-range connection, or a combination of both a short-range and along-range wireless telecommunications connection. When we refer to“short” and “long” types of connections, we do not mean to refer to thespatial relation between two devices. Instead, we are generallyreferring to short range and long range as different categories, ortypes, of connections (i.e., a primary connection and a secondaryconnection). A short-range connection may include, by way of example andnot limitation, a Wi-Fi® connection to a device (e.g., mobile hotspot)that provides access to a wireless communications network, such as aWLAN connection using the 802.11 protocol; a Bluetooth connection toanother computing device is a second example of a short-rangeconnection, or a near-field communication connection. A long-rangeconnection may include a connection using, by way of example and notlimitation, one or more of CDMA, GPRS, GSM, TDMA, and 802.16 protocols.

Many different arrangements of the various components depicted, as wellas components not shown, are possible without departing from the scopeof the claims below. Aspects of the present technology have beendescribed with the intent to be illustrative rather than restrictive.Alternative aspects will become apparent to readers of this disclosureafter and because of reading it. Alternative means of implementing theaforementioned can be completed without departing from the scope of theclaims below. Certain features and sub-combinations are of utility andmay be employed without reference to other features and sub-combinationsand are contemplated within the scope of the claims.

The invention claimed is:
 1. A method for converting a streaming videofrom a first video streaming protocol to a second video streamingprotocol, the method comprising: receiving at a server, from a computernetwork, a plurality of data packets encoded in the first videostreaming protocol, the plurality of data packets being part of acommunication session that transmits video content from a video sourceto a client device; at the server, extracting a plurality of videoframes from the plurality of data packets, the plurality of video framesencoded in a first content protocol; at the server, storing theplurality of video frames in an input buffer and performing one or bothof the following in the input buffer to produce a plurality of checkedvideo frames: (i) performing a quality check to determine whether videoframes that should be in the communication session are missing from theplurality of video frames, and (ii) temporally ordering the plurality ofvideo frames; at the server, encoding the plurality of checked videoframes in the second video streaming protocol without transcoding theplurality of checked video frames, to form a new plurality of datapackets encoded according to the second video streaming protocol withthe plurality of checked video frames being in the first contentprotocol; and outputting from the server, over the computer network, thenew plurality of data packets to the client device.
 2. The method ofclaim 1, further comprising: extracting a plurality of audio frames fromthe plurality of data packets, the plurality of audio frames encoded ina second content protocol; encoding the plurality of audio frames withinthe new plurality of data packets in the second video streamingprotocol.
 3. The method of claim 2, further comprising synchronizing theplurality of video frames to the plurality of audio frames.
 4. Themethod of claim 1, further comprising: receiving at the server, over thecomputer network, a media control signal from the client device in afirst signal protocol; at the server, converting the media controlsignal from the first mat protocol to a second signal protocol; andoutputting from the server, over the computer network, the media controlsignal to the video source in the second signal protocol.
 5. The methodof claim 4, wherein the new plurality of data packets outputted to theclient device comprise a video stream and the media control signalpauses transmission of the video stream.
 6. The method of claim 1,wherein the first and second video streaming protocols are each selectedfrom the group consisting of WebRTC (Web Real-Time Communication) andRTP (Real-Time Transport Protocol).
 7. The method of claim 1, whereinthe plurality of video frames comprise i-frames and b-frames. 8.Non-transitory computer-readable storage media havingcomputer-executable instructions for converting a streaming video from afirst video streaming protocol to a second video streaming protocolstored thereon, wherein when executed by at least one processor thecomputer-executable instructions cause the at least one processor to:receive at a server, from a computer network, a plurality of datapackets encoded in the first video streaming protocol, the plurality ofdata packets being part of a communication session that transmits videocontent from a video source to a client device; at the server, extract aplurality of video frames from the plurality of data packets, theplurality of video frames encoded in a first content protocol; at theserver, store the plurality of video frames in an input buffer andperform one or both of the following in the input buffer to produce aplurality of checked video frames: (i) perform a quality check todetermine whether video frames that should be in the communicationsession are missing from the plurality of video frames, and (ii)temporally order the plurality of video frames; at the server, encodethe plurality of checked video frames in the second video streamingprotocol without transcoding the plurality of checked video frames, toform a new plurality of data packets encoded according to the secondvideo streaming protocol with the plurality of checked video framesbeing in the first content protocol; and output from the server, overthe computer network, the new plurality of data packets to the clientdevice.
 9. The non-transitory computer-readable media of claim 8,wherein the computer-executable instructions further cause the at leastone processor to— extract a plurality of audio frames from the pluralityof data packets, the plurality of audio frames encoded in a secondcontent protocol; encode the plurality of audio frames within the newplurality of data packets in the second video streaming protocol. 10.The non-transitory computer-readable media of claim 9, wherein thecomputer-executable instructions further cause the at least oneprocessor to synchronize the plurality of video frames to the pluralityof audio frames.
 11. The non-transitory computer-readable media of claim8, wherein the computer-executable instructions further cause the atleast one processor to— receive at the server, over the computernetwork, a media control signal from the client device in a first signalprotocol; at the server, convert the media control signal from the firstsignal protocol to a second signal protocol; and output from the server,over the computer network, the media control signal to the video sourcein the second signal protocol.
 12. The non-transitory computer-readablemedia of claim 11, wherein the new plurality of data packets outputtedto the client device comprise a video stream and the media controlsignal pauses transmission of the video stream.
 13. The non-transitorycomputer-readable media of claim 8, wherein the first and second videostreaming protocols are each selected from the group consisting ofWebRTC (Web Real-Time Communication) and RTP (Real-Time TransportProtocol).
 14. The non-transitory computer-readable media of claim 8,wherein the plurality of video frames comprise i-frames and b-frames.15. A system for converting a streaming video from a first videostreaming protocol to a second video streaming protocol, the systemcomprising one or more processors individually or collectivelyprogrammed to: receive at a server, from a computer network, a pluralityof data packets encoded in the first video streaming protocol, theplurality of data packets being part of a communication session thattransmits video content from a video source to a client device; at theserver, extract a plurality of video frames from the plurality of datapackets, the plurality of video frames encoded in a first contentprotocol; at the server, store the plurality of video frames in an inputbuffer and perform one or both of the following in the input buffer toproduce a plurality of checked video frames: (i) perform a quality checkto determine whether video frames that should be in the communicationsession are missing from the plurality of video frames, and (ii)temporally order the plurality of video frames; at the server, encodethe plurality of checked video frames in the second video streamingprotocol without transcoding the plurality of checked video frames, toform a new plurality of data packets encoded according to the secondvideo streaming protocol with the plurality of checked video framesbeing in the first content protocol; and output from the server, overthe computer network, the new plurality of data packets to the clientdevice.
 16. The system of claim 15, wherein the one or more processorsare further individually or collectively programmed to— extract aplurality of audio frames from the plurality of data packets, theplurality of audio frames encoded in a second content protocol; encodethe plurality of audio frames within the new plurality of data packetsin the second video streaming protocol.
 17. The system of claim 16,wherein the one or more processors are further individually orcollectively programmed to synchronize the plurality of video frames tothe plurality of audio frames.
 18. The system of claim 15, wherein theone or more processors are further individually or collectivelyprogrammed to— receive at the server, over the computer network, a mediacontrol signal from the client device in a first signal protocol; at theserver, convert the media control signal from the first signal protocolto a second signal protocol; and output from the server, over thecomputer network, the media control signal to the video source in thesecond signal protocol.
 19. The system of claim 18, wherein the newplurality of data packets outputted to the client device comprise avideo stream and the media control signal pauses transmission of thevideo stream.
 20. The system of claim 15, wherein the first and secondvideo streaming protocols are each selected from the group consisting ofWebRTC (Web Real-Time Communication) and RTP (Real-Time TransportProtocol).